A Critical Appraisal of the Risk of Bias in Systematic Reviews and Meta-analyses Pertaining to COVID-19

Background: We critically evaluated the risk of bias in published systematic reviews (SRs) and meta-analyses (MAs) pertaining to COVID-19 using ROBIS tool. Materials And Methods: MEDLINE and Cochrane Central Library were searched for SRs/MAs on 14th May 2020, including studies of all designs describing various facets of COVID-19 in humans;no restrictions were applied for interventions, comparators, and outcomes. Two reviewers independently assessed all the SRs/MAs with ROBIS. Result(s): Out of 204 identified records, 48 SRs/MAs were included. The most frequently reviewed topics were therapy outcomes, diagnosis, and comorbidities (15, 8, and 6 papers respectively). Only 29/48(60.41%) papers had made a mention of using PRISMA or other guidelines for drafting the SR/MA. Only 5/48(10.42%) of all included SRs/MAs had low overall risk of bias as per ROBIS tool;41/48(85.42%) had high risk of bias, 2/48(4.17%) had unclear risk of bias. The highest proportion of bias was found in data synthesis and findings (30/48, 62.50% of studies had high risk of bias), followed by study identification and selection (29/48, 60.42%). The IRR for methodological quality assessment was substantial, with the Cohen's kappa values being 0.64, 0.68, 0.62, and 0.75 for domains 1-4 of ROBIS tool, and 0.66 for overall risk of bias assessment. Conclusion(s): There are serious concerns about the methodology employed to generate the results of the SRs/ MAs pertaining to COVID-19, with 'quantity' seemingly being given more importance than 'quality' of the paper.Copyright © 2022 Bentham Science Publishers.

Quality of systematic reviews/meta-analyses in coronavirus disease 2019 and venous thromboembolism: An analysis using a measurement tool to assess systematic reviews-2

BACKGROUND: Many systematic reviews (SRs) and meta-analyses (MAs) have been published regarding the incidence of venous thromboembolism (VTE) in coronavirus disease 2019 (COVID-19) patients, which is acknowledged as high. Therefore, we aimed to assess the methodological quality of any SR/MA involving VTE and COVID-19. METHOD(S): This is a meta-epidemiological study evaluating the methodological quality of SR/MA. A structured search in PubMed and Web of Science was done for all SRs/MAs in VTE and COVID-19 from inception till April 29, 2021. We evaluated the methodological quality using A Measurement Tool to Assess SRs-2 (AMSTAR-2) checklist. RESULT(S): We included 29 SRs/MAs where only one SR/MA was rated as high (3.44%), 12 (41.37%) were rated as moderate, and the rest were rated as low/critically low (55.17%). No statistical significance was observed in Spearman's correlation analysis between the rate of methodological quality and the number of authors, searching period, publication month, and publishing journal's impact factor. CONCLUSION(S): The number of published SRs/MAs in the VTE and COVID-19 subject is rising rapidly. The readers should pay attention to the low methodological quality of most published SRs/MAs. Copyright © 2022 Journal of Applied Hematology.

Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019.

BACKGROUND: Systematic reviews (SRs) provide the highest level of evidence and inform evidence-based decision making in health care. Earlier studies found association with industry to be negatively associated with methodological quality of SRs. However, this has not been investigated in SRs on vaccines. METHODS: We performed a systematic literature search using MEDLINE and EMBASE in March 2020. The results were restricted to those published between 2016 and 2019 with no language restrictions. Study characteristics were extracted by one person and checked by an experienced reviewer. The methodological quality of the SRs was assessed with the AMSTAR 2 tool by multiple reviewers after a calibration exercise was performed. A summary score for each SR was calculated. The Mann-Whitney U test and Fisher's exact test were performed to compare both groups. RESULTS: Out of 185 SRs that met all inclusion criteria, 27 SRs were industry funded. Those were matched with 30 non-industry funded SRs resulting in a total sample size of 57. The mean AMSTAR 2 summary score across all SRs was 0.49. Overall, the median AMSTAR 2 summary score was higher for the non-industry funded SRs than for the industry-funded SRs (0.62 vs. 0.36; p < .00001). Lower ratings for industry funded SRs were consistent across all but one AMSTAR 2 item, though significantly lower only for three specific items. CONCLUSION: The methodological quality of SRs in vaccination is comparable to SRs in other fields, while it is still suboptimal. We are not able to provide a satisfactory explanation why industry funded SRs had a lower methodological quality than non-industry funded SRs over recent years. Industry funding is an important indicator of methodological quality for vaccine SRs and should be carefully considered when appraising SR quality.

Clinical trials in COVID-19 management & prevention: A meta-epidemiological study examining methodological quality.

OBJECTIVE: To describe the characteristics of Covid-19 randomized clinical trials (RCTs) and examine the association between trial characteristics and the likelihood of finding a significant effect. STUDY DESIGN: We conducted a systematic review to identify RCTs (up to October 21, 2020) evaluating drugs or blood products to treat or prevent Covid-19. We extracted trial characteristics (number of centers, funding sources, and sample size) and assessed risk of bias (RoB) using the Cochrane RoB 2.0 tool. We performed logistic regressions to evaluate the association between RoB due to randomization, single vs. multicentre, funding source, and sample size, and finding a statistically significant effect. RESULTS: We included 91 RCTs (n = 46,802); 40 (44%) were single-center, 23 (25.3%) enrolled <50 patients, 28 (30.8%) received industry funding, and 75 (82.4%) had high or probably high RoB. Thirty-eight trials (41.8%) reported a statistically significant effect. RoB due to randomization and being a single-center trial were associated with increased odds of finding a statistically significant effect. CONCLUSIONS: There is high variability in RoB among Covid-19 trials. Researchers, funders, and knowledge-users should be cognizant of the impact of RoB due to randomization and single-center trial status in designing, evaluating, and interpreting the results of RCTs. REGISTRATION: CRD42020192095.

Reporting and methodological quality of COVID-19 systematic reviews needs to be improved: an evidence mapping.

OBJECTIVES: To assess the reporting and methodological quality of COVID-19 systematic reviews, and to analyze trends and gaps in the quality, clinical topics, author countries, and populations of the reviews using an evidence mapping approach. STUDY DESIGN AND SETTING: A structured search for systematic reviews concerning COVID-19 was performed using PubMed, Embase, Cochrane Library, Campbell Library, Web of Science, CBM, WanFang Data, CNKI, and CQVIP from inception until June 2020. The quality of each review was assessed using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) checklist and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. RESULTS: In total, 243 systematic reviews met the inclusion criteria, over 50% of which (128, 52.7%) were from 14 developing countries, with China contributing the most reviews (76, 31.3%). In terms of methodological quality of the studies, 30 (12.3%) were of moderate quality, 63 (25.9%) were of low quality, and 150 (61.7%) were of critically low quality. In terms of reporting quality, the median (interquartile range) PRISMA score was 14 (10-18). Regarding the topics of the reviews, 24 (9.9%) focused on the prevalence of COVID-19, 69 (28.4%) focused on the clinical manifestations, 30 (12.3%) focused on etiology, 43 (17.7%) focused on diagnosis, 65 (26.7%) focused on treatment, 104 (42.8%) focused on prognosis, and 25 (10.3%) focused on prevention. These studies mainly focused on general patients with COVID-19 (161, 66.3%), followed by children (22, 9.1%) and pregnant patients (18, 7.4%). CONCLUSION: This study systematically evaluated the methodological and reporting quality of systematic reviews of COVID-19, summarizing and analyzing trends in their clinical topics, author countries, and study populations.

Assessment of the quality of systematic reviews on COVID-19: A comparative study of previous coronavirus outbreaks.

Several systematic reviews (SRs) have been conducted on the COVID-19 outbreak, which together with the SRs on previous coronavirus outbreaks, form important sources of evidence for clinical decision and policy making. Here, we investigated the methodological quality of SRs on COVID-19, severe acute respiratory syndrome (SARS), and Middle East respiratory syndrome (MERS). Online searches were performed to obtain SRs on COVID-19, SARS, and MERS. The methodological quality of the included SRs was assessed using the AMSTAR-2 tool. Descriptive statistics were used to present the data. In total, of 49 SRs that were finally included in our study, 17, 16, and 16 SRs were specifically on COVID-19, MERS, and SARS, respectively. The growth rate of SRs on COVID-19 was the highest (4.54/month) presently. Of the included SRs, 6, 12, and 31 SRs were of moderate, low, and critically low quality, respectively. SRs on SARS showed the optimum quality among the SRs on the three diseases. Subgroup analyses showed that the SR topic (P < .001), the involvement of a methodologist (P < .001), and funding support (P = .046) were significantly associated with the methodological quality of the SR. According to the adherence scores, adherence to AMSTAR-2 items sequentially decreased in SRs on SARS, MERS, and COVID-19. The methodological quality of most SRs on coronavirus outbreaks is unsatisfactory, and those on COVID-19 have higher risks of poor quality, despite the rapid actions taken to conduct SRs. The quality of SRs should be improved in the future. Readers must exercise caution in accepting and using the results of these SRs.